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AUTOPAVER,  A  SOFTWARE  PACKAGE  FOR 
AUTOMATED  PAVEMENT  EVALUATION 


1  INTRODUCTION 


Background 

PAVER*  is  a  field-tested  and  validated  pavement  maintenance  management  system  for  use  by 
military  installations,  U.S.  Army  Corps  of  Engineers  (USAGE)  Districts,  airports,  and  civilian 
municipalities.  PAVER  is  designed  to  optimize  the  allocation  of  funds  for  pavement  maintenance  and 
repair.  PAVER’S  key  component  is  the  Pavement  Condition  Index  (PCI),  an  objective  rating  of  pavement 
condition  based  on  the  quantity  and  severity  of  observed  distress.  The  PCI  provides  a  consistent  measure 
of  a  pavement’s  structural  integrity  and  operational  condition. 

Information  about  a  pavement’s  condition  is  usually  gathered  by  someone  who  must  view  the 
pavement,  determine  which  distresses  are  present,  measure  the  size  of  the  affected  area,  and  record  all 
information  for  later  use  by  PAVER.  This  process  is  the  most  time  consuming  part  of  using  the  PAVER 
system.  In  order  to  accumulate  accurate  data,  a  field  crew  must  travel  to  the  site  and  cordon  off  the  area 
for  painstaking  measurements.  Traffic  must  be  routed  around  the  inspection  area.  The  crew  must  visually 
inspect  the  affected  areas,  classify  the  pavement  damage,  and,  using  a  tape  measure,  gauge  the  exact 
dimensions  of  each  distress.  If  the  pavement  is  cracked,  the  crew  categorizes  the  type  of  crack  and 
measures  the  length  and  width  of  the  cracked  area.  Other  types  of  distresses  demand  different  types  of 
measurement.  Holes,  for  instance,  require  measurements  for  subsequent  computation  of  surface  area.  The 
process  as  a  whole  is  not  only  slow  and  cumbersome,  but  physically  dangerous  to  the  crew,  since  it  must 
work  adjacent  to  the  flow  of  traffic.  A  method  is  needed  to  improve  the  efficiency  of  data  collection  and 
to  reduce  data  entry  times  for  PCI  surveys. 


Objectives 

The  objectives  of  this  woik  were:  (1)  to  develop  a  microcomputer  software  package  that  analyzes 
pictures  of  pavement  surfaces  and  forwards  the  resulting  information  to  PAVER,  (2)  to  describe  how 
image  processing  can  be  implemented  in  pavement  analysis,  and  (3)  to  describe  the  capabilities  of 
AutoPAVER  by  outlining  its  user  interface. 


‘  M.  Y.  Shahin,  K.  A.  Cation,  and  M.  R.  Broten,  Pavement  Maintenance  Management:  the  Micro  Paver  System,  Technical  Report 
M-87/12/ADA187360  (U.S.  Army  Construction  Engineering  Research  Laboratory  [USACERL],  July  1987);  D.  R.  Uzarski  and 
R.  C.  Soule,  The  Practical  Use  of  PAVER  in  Planning,  Programming,  and  Developing  Projects  for  Pavement  Maintenance  and 
Repair,  Technical  Report  M-86/04/ADA167312  (USACERL,  March  1986);  M.  Y.  Shahin  and  S.  D.  Kohn,  Overview  of  the 
"PAVER"  Pavement  Management  System  and  Economic  Analysis  of  Field  Implementing  the  "PAVER"  Pavement  Management 
System,  Technical  Manuscript  M-310/ADA116311  (USACERL,  January  1982). 
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Approach 


AutoPAVER  simplifies  the  workload  of  gathering  and  transmitting  data  to  the  PAVER  program. 
Pavement  sections  are  photographed  and  the  photos  are  then  fed  into  an  image-processing  system.  The 
user  works  interactively  with  the  software  program  by  categorizing  pavement  damage,  and  then  by 
identifying  and  outlining  pavement  distresses  in  the  screen  image.  AutoPAVER  uses  sophisticated 
algorithms  to  measure  the  user-defined  distresses  and  then  performs  the  necessary  data  entry. 


Mode  of  Technology  Transfer 

AutoPAVER  will  be  made  available  to  PAVER  users  through  U.S.  Army  distribution  centers. 
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2  IMAGE-PROCESSING  TECHNOLOGY 


Image  processing  is  the  science  of  modifying  and  analyzing  pictures.  The  basic  goals  of  image 
processing  include  enhancement  or  modification  of  an  image  to  improve  its  appearance  or  highlight 
information,  measurement  of  image  elements,  classification  or  matching  of  image  elements,  and 
recognition  of  items  in  an  image. 

AutoPAVER  relies  on  image-processing  techniques  to  help  analyze  and  transform  an  image  into 
useful  PAVER  information.  Image  processing  is  performed  using  step-by-step  procedures  called 
algorithms.  These  algorithms  are  often  implemented  using  computers  that  are  flexible  and  have  relatively 
low  processing  and  memory  costs.  The  algorithms  are  expressed  as,  and  become  nearly  synonymous  with, 
programs  for  the  computer.  An  algorithm  can  specify  operations  such  as  how  to  acquire  the  image. 
Special  image-processing  hardware  often  supplements  the  computer. 


Terminology  for  Image  Processing 

Before  discussing  the  details  of  image  processing,  it  is  important  to  provide  a  frame  of  reference  with 
respect  to  the  terminology  used  in  this  report.  Because  so  many  branches  of  science  have  used  image 
processing  independently,  people  have  invented  many  different  terms  that  describe  the  same  ideas.  This 
report  describes  applications  of  computer-assisted  image  processing  using  terminology  from  the  computer 
science  and  electrical  engineering  disciplines. 


Image-Processing  Hardware 

AutoPAVER  requires  electronic  equipment  suitable  for  acquiring,  processing,  and  displaying 
monochrome  images.  The  necessary  hardware  consists  of  a  camera,  a  frame  grabber  (a  digitizing  circuit 
with  a  section  of  computer  memory  dedicated  to  holding  an  image),  a  computer  that  can  access  image 
memory,  and  a  video  monitor  to  display  the  contents  of  the  memory  (i.e.,  the  picture  being  processed). 


The  digitizing  circuit  places  a  camera  image  into  the  image  memory.  This  process  involves  digitizing 
a  video  frame  by  breaking  it  into  an  array  of  digital  intensity  values  called  pixels  (short  for  "picture 
elements").  If  the  image  is  already  represented  as  pixels,  as  in  a  computer  graphic,  image  acquisition 
consists  of  simply  moving  the  image  from  disk  to  the  image  memory. 

The  computer  reads  and  writes  information  in  image  memory.  The  display  device  reads  image 
memory  and  shows  a  representation  of  it.  An  entire  video  image  stored  in  image  memory  is  often  called 
a  "frame  buffer"  or  "frame  store."  The  most  common  frame  size  is  derived  from  the  National  Television 
Standard  for  Color  (NTSC)  which  results  in  a  frame  size  of  512  by  480  pixels. 

Most  low  cost  computers  work  with  data  that  is  represented  by  eight  bits  (binary  digits).  For 
AutoPAVER,  each  pixel  has  an  eight-bit  byte,  which  yields  2*  (256)  different  shades  of  gray.  Zero 
represents  black,  255  represents  white,  and  all  the  shades  of  gray  fall  in  the  range  between  zero  and  255. 

The  computer  manipulates  the  pixels  in  the  image  memory.  The  display  device  converts  processed 
pixels  back  into  spatially  organized  image  intensities.  This  display  device  is  usually  a  digital-to-analog 
(D/A)  converter  that  drives  a  monochrome  or  color  television  monitor. 
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The  computer  used  to  develop  and  run  AutoPAVER  is  an  80386  personal  computer,  fitted  with  an 
EGA  monitor.  The  second  monitor,  used  for  graphics  viewing  and  enhancement,  is  a  Sony  PVM-1271Q. 
The  fi^e  grabber  is  Image  Technology  Inc.’s  PC  Vision  Plus,  and  the  camera  used  is  a  Sony  X6-57 
(CTD).  It  is  worth  noting  that  the  CCD  (Charge  Coupled  Device)  feature  suits  a  camera  to  this  type  of 
research,  since  it  allows  an  image  to  be  viewed  on  a  computer  monitor  for  a  great  length  of  time  without 
the  danger  of  "burning  the  image"  permanently  into  the  screen  display.  The  pointing  device  used  in  this 
study  is  a  Microsoft  Mouse.^ 


Classifying  Algorithms 

Image-processing  algorithms  can  be  classified  in  several  ways.  If  an  algorithm  changes  a  pixel’s 
value,  it  is  c^ed  a  "point  process."  If  the  algorithm  changes  a  pixel’s  value  based  on  the  values  of  that 
pixel  and  neighboring  pixels,  it  is  called  an  "area  process."  If  the  algorithm  changes  the  position  or 
arrangement  of  the  pixels,  it  is  called  a  "geometric  process."  Algorithms  that  change  pixel  vdues  based 
on  a  comparison  of  two  or  more  images  are  called  "frame  processes"  (because  individual  video  images 
are  called  "frames"). 

Image  measurement  is  based  on  some  broad  assumptions  about  what  items  will  appear  in  the  picture. 
Gassirication  of  objects  is  relatively  simple:  recognition  requires  more  knowledge  (and  assumptions)  about 
what  can  appear  in  an  image.  For  example,  the  number  of  pixels  in  an  image  of  pavement  within  a  certain 
range  of  values  can  simply  be  counted.  If  it  is  known  that  bleeding  asphalt  corresponds  to  these  values, 
the  image  can  be  classified  into  bleeding  and  nonbleeding  areas.  If  the  machine  is  provided  with  further 
knowledge  about  the  structure  of  bleeding  asphalt,  it  might  be  able  to  recognize  these  areas  in  the  image. 
Color  may  also  be  used  to  highlight  the  recognized  areas. 

Based  on  this  classification  fiameworic  and  the  AutoPAVER  hardware,  some  example  algorithms  will 
be  described.  These  examples  are  not  necessarily  exhaustive;  they  are  intended  only  to  facilitate  an 
understanding  of  the  image-processing  options  for  this  technology. 


Point  Processes 

A  point  process  algorithm  scans  through  the  image  area  using  the  pixel  value  at  each  point  to 
compute  a  new  value  at  that  point. 

Optical  Negative 

To  produce  the  i^iotographic  negative  of  an  image,  the  computer  is  programmed  to  take  each  pixel 
value  f  (0  <=  f  <=  255)  and  replace  it  with  g  (g  =  255  -  f). 

Brightness 

Point  processes  also  can  be  used  to  enhance  or  modify  pixel  values.  For  example,  adding  40  to  each 
pixel  value  brightens  the  image  and  could  improve  the  picture’s  appearance. 

If  the  pixel  value  and  its  location  are  used,  then  a  point  process  can  be  used  to  correct  shading  or 
smoothly  change  pixel  values  in  an  image  area.  Shading  is  an  image  artifact  caused  by  slow  spatial  shifts 


*  Microsoft  is  a  registered  trademark  of  the  Microsoft  Corp,  Redmond,  WA. 
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in  scene  lighting  or  camera  bias  and  sensitivity.  A  point  process  that  computes  the  inverse  of  a  shading 
function  can  eliminate  or  correct  much  of  this  shading. 

By  smoothly  changing  the  pixel  values  in  an  area,  the  contrast  of  areas  can  be  highlighted  or 
adjusted.  This  function  can  produce  results  similar  to  the  photographic  darkroom  techniques  of  burning 
and  dodging  (methods  of  adjusting  local  contrast). 

Histogram  Stretching 

To  calculate  an  "intensity  histogram,"  the  number  of  times  a  particular  pixel  value  occurs  in  an  image 
is  counted.  Using  AutoPAVER  hardware  based  on  eight-bit  pixels,  256  pixel  values  are  possible.  An 
image  can  be  scanned  and  the  number  pixels  with  a  given  value  counted.  The  result  can  be  stored  as  one 
entry  in  a  256  place  table.  Such  a  table  is  called  a  "histogram." 

The  histogram  is  a  type  of  image  measurement.  Because  it  examines  a  single  pixel  at  a  time,  the 
histogram  is  the  result  of  a  point  process  that  leaves  the  pixel’s  value  unchanged. 

Information  provided  by  the  histogram  is  useful  for  image  enhancement  and  classification.  If  all 
pixel  values  are  bunched  in  a  small  range  (making  the  picture  appear  featureless),  this  information  can  be 
used  to  improve  image  contrast.  Starting  at  intensity  zero,  the  histogram  can  be  searched  for  the  first  pixel 
value  with  more  than  a  specified  number  of  pixel  counts.  One  might  choose  a  value  occurring  in  only 
a  small  area  of  the  screen,  say  30.  Next,  a  similar  search  is  performed  starting  at  the  highest  pixel  value. 
The  region  of  the  histogram  between  these  two  (low  and  high)  values  accounts  for  most  of  the  pixels  in 
the  image.  Then  a  point  process  is  performed  that  sets  pixel  values  below  the  low  value  to  zero  and  above 
the  high  value  to  255.  This  is  sometimes  called  a  "histogram  clip."  The  pixels  with  intermediate  values 
are  adjusted  so  that  they  span  the  range  of  zero  to  255. 

Histogram  stretching  is  a  simple  form  of  contrast  enhancement.  Note  that  the  image  has  lost  some 
information-the  pixel  values  below  the  low  value  and  above  the  high  value  have  been  set  to  constants. 
In  general,  image-processing  operations  lose  some  information  in  return  for  selecting  or  accentuating  other 
information. 

Notice  that  the  histogram  stretching  algorithm  uses  three  simpler  algorithms:  a  histogram,  a 
histogram  clip,  and  a  point  process.  Most  algorithms  are  compounds  of  other  algorithms.  Therefore,  the 
program  writer  must  know  which  algorithms  to  apply  and  in  what  order  to  apply  them  to  reach  a 
processing  goal. 

Pseudocoloring 

Pseudocoloring  of  a  monochrome  image  is  another  example  of  a  point  process.  In  this  case,  the  pixel 
value  is  the  argument  (input)  for  three  different  functions,  and  the  output  of  these  functions  drives  the  red, 
green,  and  blue  guns  of  a  color  monitor.  This  process  allows  a  monochrome  image  to  be  colored  by 
substituting  any  color  for  a  particular  shade  of  gray.  Using  eight-bit  pixels,  256  colors  can  be  displayed 
at  once. 


Area  Processes 

An  area  process  uses  information  from  neighboring  pixels  to  modify  pixel  values  or  assert  the 
existence  of  some  property  of  the  neighborhood.  Area  processes  can  generate  a  wide  range  of  effects: 
spatial  filtering  (such  as  filtering  out  repeated  elements),  changing  an  image’s  structure,  or  "sharpening" 
the  image’s  appearance  by  accentuating  intensity  changes.  Other  effects  include:  finding  objects  by 
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matching  images,  measui'iig  image  properties,  making  assertions  about  object  edges  in  the  image, 
removing  noise,  and  rring  or  smoothing  the  image. 

Convolution 

Convolution  is  a  classic  image-processing  algorithm  commonly  used  for  spatial  filtering  and  finding 
ima:^e  features.  SiiKe  convolution  requires  many  repeated  calculations,  arxl  hence  a  great  deal  of  computer 
memory,  some  subtle  implementation  issues  must  be  considered. 

The  convolution  operation  replaces  a  pixel’s  value  with  the  sum  of  that  pixel’s  value  plus  those  of 
its  neighbors,  each  wei^ted  (multiplied)  by  a  factor.  The  weighting  factors  are  called  the  "convolution 
kernel."  The  programmer  labels  the  image  points  p  (i,  j)  and  the  kernel  (weighting)  points  k  (x,y),  where 
X  and  y  range  over  values  representing  the  relative  placement  of  neighboring  pixels. 

Choose,  as  an  example,  a  neighborhood  consisting  of  a  pixel  plus  its  immediate  eight  neighbors  (left, 
right,  up,  down,  and  diagonals).  Label  the  center  pixel  as  p  (1,1).  This  center  pixel  is  replaced  by  the 
linear  sum  of  its  neighbors  times  their  respective  weighting  factors,  k. 


p(l,l)  =  p(0,0)  X 
+  p(0.1)  X 
+  p(0,2)  X 


k(0,0)  +  p(l,0) 
k(0,l)  +  p(l,l) 
k(0,2)  +  p(l,2) 


X  k(l,0) -t- p(2,0) 
X  k(l,l)-hp(2,l) 
X  k(l,2)  +  p(2,2) 


X  k(2,0) 
X  k(2,l) 
X  k(2,2) 


[Eq  1] 


To  convolve  an  image,  this  operation  is  repeated  at  every  pixel  position  in  the  image.  This  process 
is  like  sliding  a  kernel  matrix  over  each  row  of  pixels  in  the  image  matrix,  and  at  each  point,  multiplying 
the  kernel  values  with  the  image  value  "under"  it,  summing  the  result,  and  then  replacing  the  pixel  at  the 
center  of  the  kernel  with  that  value.  The  equation  then  becomes: 


P(i.  j)  =  P(i-lJ-l)  X  k(0,0)  +  p(i,  j-1)  X  k(l,0)  +  p(i+l,  j-1)  x  k(2,0)  [Eq  2] 

+  p(i-l,j)  xk(0,l)+  p(i,  j)  xk(l,l)+  p(i+l,j)  x  k(2,l) 

+  p(i-l,  j-Hl)  X  k(0,2)  +  p(i,  j+1)  X  k(l,2)  +  p(i+l,  j+1)  x  k(2,2) 


Implementation  Issues 

There  are  several  interesting  issues  to  be  met  here:  Convolving  an  area  of  size  X  by  Y  with  a  kernel 
of  size  N  by  M  requires  X  x  Y  x  N  x  M  multiplications  and  additions.  Thus,  a  512  by  480  image  with 
a  three  by  three  kernel  requires  2,21 1,840  multiplication/addition  operations;  this  process  can  take  a  long 
time  on  a  computer  without  fast  multiplication  hardware. 

If  the  kernel  is  scanned  over  the  image  and  replaces  only  the  value  under  the  center  of  the  kernel 
at  a  given  position,  what  happens  to  the  edges  of  the  image?  For  example,  with  the  three  by  three  kernel, 
a  one-pixel  border  (box)  is  left  around  the  image  where  pixels  are  not  replaced.  The  convolution  will 
always  leave  a  border  of  "garbage"  equal  to  half  the  kernel  size  around  the  image.  This  border  of  garbage 
may  be  ignored,  set  to  zero,  or  have  the  nearest  meaningful  value  copied  into  it. 
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The  convolution  on  any  pixel  could  result  in  a  value  larger  than  can  be  held  in  a  pixel-possibly  as 
large  as  the  number  of  kernel  elements  times  the  number  of  bits  in  a  pixel.  In  the  generic  system 
described  previously,  for  example,  that  would  be  3  x  3  x  256  =  2304,  for  which  12  bits  per  pixel  would 
be  needed-but  there  are  only  eight  bits  per  pixel.  Enough  accuracy  in  the  calculations  must  be  kept  to 
allow  for  this  range.  The  convolution  result  can  be  scaled  (e.g.,  each  result  divided  by  two)  if  it  is  to  go 
back  into  the  image  memory. 

In  a  related  issue,  kernel  values,  and  therefore  the  convolution  output,  can  be  positive  or  negative. 
Negative  intensity  is  mathematically  useful,  but  since  the  program  only  recognizes  values  between  zero 
and  255,  negative  intensities  will  not  generate  a  screen  display.  The  option  to  modify  the  convolution 
output  desired  to  allow  only  positive,  negative  (negated  to  positive  values),  absolute  values,  or  signed 
values  as  output.  This  also  means  that  an  additional  sign  bit  must  be  kept  in  the  calculations.  Thus,  for 
eight-bit  pixels  and  a  three  by  three  kernel,  12  bits  for  the  sum  and  an  additional  bit  for  the  sign  are 
needed. 

Convolution  itself  is  relatively  simple,  but  the  implementation  issues  complicate  it.  Unfortunately, 
this  condition  is  true  of  most  image-processing  algorithms.  For  example,  the  issues  of  internal  accuracy 
and  what  to  do  at  the  edge  of  the  image  appear  in  most  other  area  processes.  Any  useful  program  must 
address  these  issues  to  implement  and  use  the  algorithms  effectively. 

Matched  and  Spatial  Filters 

Convolution  applies  to  image  processing  by  using  matched  or  spatial  filters.  In  a  matched  filter,  the 
convolution  kernel  is  essentially  a  small  image  of  what  is  to  be  amplified  or  detected.  For  example,  an 
edge  is  detected  in  an  image  by  its  sudden  increase  or  decrease  in  image  intensity.  A  convolution  kernel 
for  detecting  a  vertical  edge  itself  looks  somewhat  like  a  vertical  edge  (Figure  1). 


-1  0  1 
-1  0  1 
-1  0  1 
-1  0  1 
-1  0  1 


Figure  1.  A  kernel  matrix  to  detect  a  vertical  edge. 


Note  the  effect  of  the  negative  values:  in  a  uniform  image  area,  where  all  pixel  values  input  to  the 
convolution  are  the  same,  the  convolution  output  will  be  zero  (since  the  sum  of  any  number  times  each 
of  the  15  kernel  elements  is  zero).  The  kernel  has  been  padded  with  a  vertical  row  of  zeros  to  make  it 
an  odd  size  in  both  directions,  so  that  it  will  detect  both  increases  and  decreases  in  intensity. 


II 


A  similar  kernel  for  amplifying  horizontal  edges  would  resemble  a  horizontal  edge  (Figure  2). 


-1  -1  -1  -1  -1 
0  0  0  0  0 
11111 


Figure  2.  A  kernel  matrix  to  detect  a  horizontal  edge. 


Larger  kernels  may  include  patterns  that  match  similar  patterns  in  the  image  (e.g.,  for  the  letter  A). 
In  this  case,  the  kernel  is  a  "template."  Detection  usually  involves  amplification  of  the  desired  feature 
followed  by  a  yes/no  question  that  asks.  "Is  the  result  above  or  below  a  certain  threshold  point?" 

In  a  spatial  filter,  the  convolution  kernel  reflects  wave  frequencies.  Sound  frequency  is  a  count  of 
the  number  of  times  per  second  a  wave  form  repeats.  Image  frequency  is  the  breakdown  of  an  image  into 
a  series  of  sine  and  cosine  waves.  This  breakdown  can  be  done  using  a  fast  Fourier  transform.  The 
transform  should  be  done  both  horizontally  and  vertically  to  reflect  spatial  frequencies  in  both  directions. 

A  kernel  can  be  built  to  select,  and  peihaps  detect,  a  certain  band  of  frequencies.  Quickly  changing 
image  intensities  are  represented  by  high  spatial  frequencies,  and  slowly  changing  intensities,  by  lower 
spatial  frequencies. 

The  following  kernel  could  be  used  to  select  high  spatial  frequencies  (Figure  3). 


-1  -1  -1 
-1  8  -1 
-I  -1  -1 

Figure  3.  A  kernel  matrix  to  select  high  spatial  frequencies. 


This  "Laplacian  filter"  approximates  an  unoriented  second  derivative  operation  (i.e.,  it  measures  a 
wavelength,  the  center  frequency  of  which  is  no  more  than  twice  as  large  as  the  kernel).  Because  edges 
have  high  spatial  frequencies  (sudden  intensity  changes),  this  kernel  selects  edges  of  any  direction  and 
might  be  used  as  an  "edge  detector"  for  image  analysis. 

If  the  Laplacian  kernel  is  slightly  modified  by  making  the  center  kernel  element  nine  instead  of  eight, 
the  filter  would  add  the  output  of  the  Laplacian  convolution  to  the  original  image  (since  a  kernel  with  a 
one  in  the  middle  surrounded  by  zeros  would  yield  the  source  image  unchanged).  This  kernel  selectively 
boosts  high  frequencies  (edges)  making  the  resulting  image  sharper  and  noisier.  On  the  other  hand,  if  a 
kernel  were  used  that  matches  lower  spatial  frequencies,  the  image  will  be  blurred. 

The  power  of  convolution  lies  in  using  information  in  a  large  area  to  make  assertions  about  some 
property  at  individual  image  points.  For  example,  the  edge  operators  above  sharpen  the  edges  in  an  image 
by  using  the  fact  that  physical  edges  are  narrow  and  extend  over  some  distance.  The  art  of  convolving 
lies  in  designing  the  right  kernel,  using  a  conjunction  of  experience  and  theory. 
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Nonlinear  Area  Processes 


Convolution  is  relatively  easy  to  implement,  use,  and  analyze  because  it  is  a  linear  operation:  it 
requires  only  sums  of  first  degree  products.  Nonlinear  operations,  while  a  bit  more  difficult,  are  also 
useful  and  can  be  more  powerful  than  a  convolutioa  "Powerful"  means  either  that  they  provide  a  better 
signal-to-noise  ratio  for  detecting  image  elements  or  that  they  can  detect  features  with  less  computation. 
Consider  the  following  two  examples  of  nonlinear  area  processes. 

Sobel  Filtering 

A  Sobel  filter  compares  the  result  of  two  convolutions.  The  first  convolution  computes  the  degree 
to  which  an  edge  is  oriented  in  the  X  direction,  while  the  second  computes  the  same  for  the  Y  direction. 
Simple  trigonometry  is  used  to  estimate  the  strength  and  orientation  of  edge.  The  two  kernels,  X  and  Y, 
are  shown  in  Figure  4. 


-10  1  12  1 
X:  -2  0  2  Y:  0  0  0 
-10  1  -1  -2-1 

Figure  4.  Sobel  filter  kernel  matrices. 


Thus,  the  edge  strength  and  orientation  are  represented  by: 


Strength  =  sqtt(X*  +  Y®) 


Orientation  =  arctan  (Y/X) 


[Eq  3] 


This  is  a  first  derivative  (oriented)  edge  finder,  and  the  vector  field  it  produces  cannot  be  directly 
shown  on  a  two  dimensional  image.  The  Sobel  is  a  good  edge  detector  and  is  frequently  used  as  the  first 
step  in  machine-vision  algorithms.  Because  the  Sobel  algorithm  uses  a  lot  of  computer  memory,  various 
approximations  have  been  developed  to  implement  it 

Median  Filtering 

A  median  filter  replaces  the  pixel  at  the  center  of  a  neighborhood  of  pixels  with  the  median  of 
the  neighborhood  pixel  vdues.  The  neighborhood  values,  including  the  center  pixel,  are  sorted  into 
ascending  order,  and  the  median  (middle)  value  is  used  to  replace  the  center  pixel.  The  effect  of  a  median 
filter  is  to  remove  spot  and  low  level  noise  while  retaining  larger  scale  image  features. 


Pattern  and  Object  Identification 

Before  it  can  classify  image  elements,  the  computer  must  first  contain  a  definition  of  "image 
element"  This  definition  can  be  elaborate,  but  a  simple  example  will  suffice:  an  element  is  a  connected 
group  of  pixels.  A  "connected"  pixel  has  the  same  value  as  a  neighbor  at  0,  90,  180,  or  270  degrees. 
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To  simplify  the  computation,  the  image  can  be  "binarized";  that  is,  all  pixel  values  above  zero  and 
below  255  can  be  converted.  This  is  often  done  in  machine  vision  since  the  process  can  control  the 
lighting  and  objects  in  the  image.  The  image  is  searched  from  top  to  bottom  to  locate  elements  with  more 
than  N  (number  of)  connected  pixels.  The  specified  minimum  of  N  pixels,  eliminates  the  small  spots  of 
noise  introduced  by  thresholding.  Each  element  is  labeled  by  changing  its  pixel  value  to  a  predetermined 
number.  The  number  of  pixels  in  an  element  can  also  be  recorded  for  further  classification. 


Geometric  Processes 

Geometric  processes  change  the  spatial  arrangement  of  pixels.  They  are  often  used  to  correct  for 
distortions  caused  by  camera  optics  or  viewpoint.  They  also  can  enlarge  a  particular  image  area.  Typical 
geometric  processes  rotate,  stretch,  and  translate  the  image  position.  Other  geometric  processes  can  warp 
the  image.  Geometric  algorithms  can  be  expressed  by  a  set  of  equations  (a  matrix)  that  maps  a  pixel  at 
location  x,y  into  a  new  address  x’,y’.  For  example,  to  rotate  a  square  area  clockwise  by  90  degrees,  the 
pixels  are  mapped  by  the  following  equation. 


X’  =  (512  -  1)  -  y;  y’  =  x 


[Eq  4] 


Because  digital  pixels  are  oriented  in  a  strict  checkerboard  pattern,  most  geometric  transforms  are 
found  to  have  gaps  between  the  output  pixels.  If  source  pixels  are  placed  in  the  destination  area  according 
to  the  transformatiop  equations,  the  exact  placement  in  the  output  image  is  rarely  an  integer.  (This  is  one 
problem  with  digitized  images  that  never  occurs  in  continuous  tone  images  such  as  photographic  prints.) 


Frame  Processes 

Algorithms  that  use  more  than  one  image  are  sometimes  called  "frame  processes."  A  simple 
example  is  to  subtract  one  image  from  another.  The  resulting  differences  can  be  used  to  compare  the  two 
images  (e.g.,  to  look  for  missing  parts  on  a  machine  or  circuit  board,  or  to  look  at  the  same  piece  of 
pavement  photographed  several  years  apart  to  see  the  progression  of  damage).  Frame  processes  can  also 
be  used  to  improve  image  quality  and  to  detect  motion. 

If  a  television  camera  views  a  still  object,  a  predefined  number  (N)  of  successive  image  frames 
can  be  summed  to  reduce  noise  introduced  by  the  camera.  This  process  requires  a  frame  memory  with 
enough  bits  per  pixel  to  accommodate  the  sum.  Dividing  the  sum  by  N  produces  an  averaged  image.  If 
the  noise  is  not  correlated  from  frame  to  frame,  the  improvement  in  signal  to  noise  will  be  of  the  order 
sqrt(N).  A  typical  low  cost  video  camera  has  about  three  bits  of  noise,  so  that  averaging  with  an  N  of 
eight  or  16  will  noticeably  improve  the  image. 
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3  AUTOPAVER  FEATURES 


AutoPAVER  Menu  Options 

AutoPAVER  includes  a  menu  driven  interface  to  guide  the  user  through  its  various  functions. 
Upon  invoking  AutoPAVER,  the  user  sees  a  Main  Menu  bar  across  the  top  of  the  screen  containing  the 
eight  Main  Menu  commands:  PICTURE,  LUTS,  FILTERS,  CONVOL,  VIDEO,  AUTOPAVER, 
MACROS,  and  CONFIG.  By  placing  the  mouse  pointer  on  any  one  of  the  main  menu  commands  and 
depressing  and  holding  the  mouse  button,  the  user  calls  a  pop  up  window  that  extends  downward  from 
the  selected  main  menu  entry,  and  that  lists  the  submenu  elements  for  that  main  menu  command.  By 
dn*"  ing  the  mouse  pointer  to  the  appropriate  command  and  releasing  the  mouse  button,  the  user  selects 
iLi^vant  AutoPAVER  operations.  Table  1  shows  each  Main  Menu  command  with  its  corresponding 
submenu. 


Table  1 

Main  and  Submenu  Commands 


PICTURE 

LUTS 

FILTERS 

CONVOL 

VIDEO 

AUTOPAVER 

MACROS 

coNnc 

Quadrant 

P  Pseudocolor 

High  Pass 

Sharpen 

OffsetASain 

AP  Load 

Load 

Save  Global 

Load 

M  Pusudocolor 

Low  Pass 

Average 

Vertical  Wedge 

APSave 

Save 

Save  Local 

Save 

Negative 

Band  Pass 

Haiinntal 

Horizontal  Wedge 

AP  Save  At 

Save  Aa 

Read  Global 

Save  As 

Hist  Stretch 

Band  Cut 

Laplacian 

Upside  down 

Send 

WaichMe 

Read  Local 

Print 

Hist  Plot 

Binarize 

Lap4 

Grab 

Send  To 

Flay  Back 

Read  Path 

Quit 

Britten 

Resioie 

Median 

Snap 

Change  Pvmnt 

Picture  l^ih 

About  AutoPAVER  Dim 

Edge 

Qear  Sheet 

AutoPAVER  Path 

Unify 

Sobel 

Dimensions 

PAVER  Path 

Uneinze 

PAVER  TooU 

Macro  path 

Drawing  Tools 

The  following  section  lists  the  AutoPAVER  menu  options  with  a  brief  description 
of  each. 

PICTURE 

About  AutoPAVER  -  Lists  program’s  developers  and  gives  point  of  contact. 

Quadrant  -  Menu  to  choose  which  quadrant  that  "Unify"  and  CONVOL  will  modify. 
Load  -  Menu  to  load  picture  from  disk. 

Save  -  Saves  picture  to  disk  in  picture  file  last  used. 
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Save  As  -  Saves  picture  to  disk  under  a  new  name. 


Print  -  Sends  picture  to  printer.  Currently  supports  Epson-FX,  which  is  a  two  color  printer. 
Consequently  it  is  best  to  use  Binarize  to  adjust  how  the  picture  will  look  and  then  Unify  to  change 
frame  store  before  printing. 

Quit  -  Prompts  to  leave  program. 


LUTS  (Changes  Look  Up  Tables  only) 

P  Pseudocolor  -  One  method  of  pseudocoloring.  Use  Horizontal  Wedge  or  Vertical  Wedge  to  explore 
how  this  works. 

M  Pseudocolor  -  A  second  method  of  pseudocoloring.  Use  Horizontal  Wedge  or  Vertical  Wedge  to 
explore  how  this  works. 

Negative  -  Produces  an  optical  negative. 

Hist  Stretch  -  Histogram  Stretch  (explained  above  in  section  2)  causes  overall  picture  to  have  better 
contrast 

Hist  Plot  -  Posts  up  histogram  of  current  quadrant  of  the  screen  in  the  current  quadrant  of  the  screen. 
This  is  arranged  from  zero  (black)  on  the  left  to  255  (white)  on  the  right. 

Brighten  -  Increases  all  gray  values. 

Dim  -  Decreases  all  gray  values. 

Unify  -  Copies  LUTS  values  onto  frame  store  (includes  filter  effects). 

Linearize  -  Qears  LUTS  tables  to  normal  black  and  white. 


FILTERS  (Changes  look  up  tables  only) 

High  Pass  -  Leaves  whiter  values  (above  adjustable  cutoff)  untouched  and  maps  darker  values  onto 
chosen  replacement  color. 

Low  Pass  -  Leaves  darker  values  (below  adjustable  cutofO  untouched  and  maps  lighter  values  onto 
chosen  replacement  color. 

Band  Pass  -  Leaves  middle  gray  levels  (between  adjustable  cutoffs)  untouched  and  maps  higher  and 
lower  values  onto  replacement  color. 

Band  Cut  -  Leaves  extreme  gray  levels  (outside  adjustable  cutoffs)  untouched  and  maps  middle  levels 
onto  replacement  color. 

Binarize  -  Forces  all  colors  (either  side  of  adjustable  cutoff)  to  be  mapped  to  black  and  white. 

Restore  -  Removes  filter  effects. 
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CONVOL  (Convolutions) 

Sharpen  -  Causes  image  to  appear  sharper  by  local  contrast  stretching.  (User  selects  degree  of 
stretching.) 

Average  -  Causes  image  to  appear  fuzzier  (or  less  noisy)  by  local  contrast  averaging. 

Vertical  -  Vertical  edge  detector.  Leaves  bright  areas  near  vertical  edges  and  darkens  other  areas. 

Horizontal  -  Horizontal  edge  detector.  Leaves  bright  areas  near  horizontal  edges  and  darkens  other 
areas. 

Laplacian  -  Edge  detector  approximates  the  magnitude  of  the  second  derivative. 

Lap4  -  Edge  detector  sensitive  to  horizontal  and  vertical  edges.  An  alternate  form  of  Laplacian. 

Median  -  Takes  median  of  pixel  neighborhood  (user  selectable). 

Edge  -  Edge  detector  approximates  result  from  Sobel  kernel  (below)  only  faster 

Sobel  -  Commonly  used  edge  detector.  Leaves  thick  indication  lines  thereby  allowing  easier 
segmentation  of  the  image. 

VIDEO 

OffsetIGain  -  Allows  user  adjustment  of  offset  and  gain  of  the  camera.  (Offset  and  gain  arc  analogous 
to  the  brightness  and  contrast  knobs  of  a  television  set.) 

Vertical  wedge  -  Standard  test  pattern. 

Horizontal  wedge  -  Standard  test  pattern 

Upside  dawn  -  Turns  image  upside  down. 

Grab  -  Continuous  view  from  camera. 

Snap  -  Snaps  a  picture  with  the  camera. 

AUTOPAVER 

AP  Load  -  Menu  to  load  AutoPAVER  analysis  file. 

AP  Save  -  Saves  AutoPAVER  analysis  file  under  analysis  file  name  last  used. 

AP  Save  As  -  Prompts  for  name  of  file  and  store  AutoPAVER  analysis. 

Send  -  Sends  PAVER  data  to  filename  last  used. 

Send  To  -  Selects  new  filename  and  send  PAVER  data  to  that  file. 
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Change  Pvmrit  -  Toggles  pavement  type  (asphalt/concrete). 

Clear  Sheet  -  aears  all  analysis  previously  drawn  by  user. 

Dimensions  -  Sets  the  dimensions  of  the  image  in  the  video  screen. 

PAVER  Tools  -  Posts  the  PAVER  distress  types  for  current  pavement  type. 
Drawing  Tools  -  Posts  up  the  various  drawing  tools. 

MACROS  (For  often  repeated  operations.) 

Load  -  Menu  to  select  macro  file. 

Save  -  Saves  current  macro  set  to  macro  file  last  used. 

Save  As  -  Prompts  user  for  macro  file  name  and  save  current  macros  there. 
Watch  Me  -  Starts  macro  recording. 

Play  Back  -  Repeats  a  macro. 


CONFIG 

Sccve  Global  -  Saves  the  current  configuration  to  the  global  configuration  file  (i.e.,  the  configuration 
used  when  AutoPAVER  is  executed  from  any  drive  or  directory  other  than  those  with  a  local 
configuration). 

Save  Local  -  Saves  the  current  configuration  to  the  local  configuration  file  (i.e.,  the  configuration  used 
when  AutoPAVER  is  executed  from  that  particular  drive/directory). 

Read  Global  -  Loads  the  global  configuration  file. 

Read  Local  -  Loads  the  local  configuration  file. 

Picture  Path  -  Selects  drive  and  path  where  picture  files  are  kept. 

AutoPAVER  Path  -  Selects  drive  and  directory  where  AutoPAVER  analysis  files  are  kept. 

PAVER  Path  -  Selects  drive  and  path  where  PAVER  data  files  kept. 

Macro  Path  -  Selects  drive  and  path  where  macro  files  are  kept. 


Using  AutoPAVER 

The  AutoPAVER  system  has  the  appearance  and  simplicity  of  a  coloring  book.  The  system  requires 
two  video  displays,  one  for  the  AutoPAVER  menu  interface,  and  the  other  for  the  graphic  display  of 
pavement.  On  the  first  screen,  the  user  selects  commands  and  AutoPAVER  operations  that  will  affect  the 
video  display  on  the  second  screen.  When  the  user  slides  the  mouse  pointer  past  the  edge  of  the  menu 
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screen,  the  pointer  reappears  on  the  graphics  screen.  In  this  way,  the  user  travels  between  the  two  screens, 
using  the  mouse  pointer  in  the  program  screen  to  select  commands,  and  then  in  the  graphic  screen,  to 
outline  distressed  areas  of  the  displayed  section  of  pavement.  The  user  views  a  photograph  of  a  pavement 
section  on  the  graphics  monitor,  indicates  to  the  system  what  type  of  pavement  distress  appears  in  the 
picture,  and  marks  the  distress  (on  the  video  screen)  with  a  pointing  device  (i.e.,  a  mouse).  The  system 
performs  all  numerical  calculations  and  automatically  forwards  this  information  to  PAVER.  The  user 
makes  no  numerical  measurements  or  calculations,  and  needs  no  manual  numerical  record  keeping. 

Using  the  CONFIG  submenu,  the  user  must  enter  the  following  information  into  AutoPAVER  in 
order  for  the  system  to  be  properly  configured: 

1.  The  base  memory  address  of  the  frame  grabber 

2.  The  base  I/O  address  of  the  fixune  grabber 

3.  The  picture  drive/directory 

4.  The  AutoPAVER  analysis  drive/directory 

5.  The  PAVER  data  drive/directory 

6.  The  Macro  file  drive/directory. 

One  subtlety  in  using  AutoPAVER  is  that  sometimes  the  data  which  makes  up  the  frame  buffer  (how 
the  computer  sees  the  picture)  is  different  than  that  which  is  shown  on  the  data  analysis  screen.  Whenever 
this  happens,  there  will  be  a  warning  indicator  on  the  second  line  of  the  computer  screen. 

Many  times  this  is  a  desirable  situation.  It  helps  user  to  "try  out"  certain  ways  of  displaying  the  data 
without  changing  the  frame  buffer.  To  delete  the  warning  indicator,  simply  use  the  mouse  pointer  to 
click  on  the  warning  indicator  icon,  and  a  computer  will  prompt  to  change  the  frame  buffer  from  the  way 
the  screen  looks  to  a  simple  display  of  the  frame  buffer  as  Ae  computer  sees  it  (e.g.,  "Change  Frame 
Buffer?").  Simply  choose  which  display  you  want  and  the  computer  will  take  care  of  the  rest 
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4  CONCLUSIONS 


A  microcomputer  software  package,  AutoPAVER,  was  developed  that  analyzes  pictures  of  pavement 
surfaces  and  forwards  the  resulting  information  to  the  PAVER  pavement  management  system. 
AutoPAVER  accomplishes  pavement  analysis  through  the  use  of  image-processing  techniques  and 
algorithms.  The  AutoPAVER  system  uses  a  commercially  available  computer  equipment  and  is  accessed 
through  a  menu-driven  software  interface. 

The  AutoPAVER  system  saves  both  time  and  labor  since  it  relieves  the  user  of  dangerous  traffic¬ 
blocking  chores  while  examining  pavement,  and  because  it  performs  all  numerical  tallying  and  record 
keeping  unassisted.  Development  of  various  methods  for  automatic  detection  of  pavement  distresses  is 
under  way  and  will  be  added  to  AutoPAVER  as  each  becomes  available. 

AutoPAVER  is  beneficial  as  an  intermediate  step  between  a  totally  manual  field  inspection  method 
and  a  totally  automated  image-processing  system.  7be  system  offers  three  principal  advantages: 

1.  Data  entry  time  is  eliminated  and  the  time  required  to  measure  distresses  is  reduced. 

2.  There  is  no  need  to  control  or  block  traffic  for  this  procedure  since  available  commercial 
equipment  can  take  pavement  photographs  at  the  speed  of  traffic. 

3.  A  record  of  the  pavement  condition  is  kept  on  either  35  mm  film  or  video  tape.  This  will  be 
useful  for  verifying  data  and  also  for  future  research. 
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